System for sensing and responding to a lateral blind spot of a mobile carrier and method thereof

ABSTRACT

The present application is to provide a system for sensing and responding to a lateral blind spot of a mobile carrier and method thereof, which is applied for a mobile carrier during moving to a parking place. Firstly, a light scan unit and a depth image capture unit are used to scan a plurality of surrounding objects and capture a plurality of object depth images of the surrounding objects, and then a plurality of screened images are obtained according to a moving route of the mobile carrier for further obtaining correspondingly a plurality of forecasted lines to generate corresponded notice message for noting driver or ADAS. Due to the objects corresponding to the screened images and located on a blind position which is at one side of the mobile carrier, the notice message provides the driver preventing from the ignored danger by ignoring the blind position.

FIELD OF THE INVENTION

The present application relates generally to a warning sensor fusionsystem and the method thereof, and particularly to a mobile carrierwarning sensor fusion system and the method thereof.

BACKGROUND OF THE INVENTION

Traditional advanced driver assistance systems (ADAS) are developed toassist drivers and can be divided into three main parts: automotivesensors, automotive processors, and actuators. ADAS sense signalsoutside carriers using automotive sensors such as millimeter-waveradars, lidar, thermal sensors, and pressure sensors. The sensing datafrom automotive sensors are transmitted to automotive processors, forexample, electronic control units (ECU), for producing warninginformation for drivers according to the sensing data and thus avoidingdangerous road conditions. Furthermore, automotive sensors can evenintervene the drivers' driving operations directly and activatingactuators for slowing, emergency braking, or turning cars and protectingdrivers.

In addition, to protect drivers, radar detection technologies aredeveloped to detect carrier surroundings. Unfortunately, radars cannotdifferentiate fixed or mobile objects surrounding a carrier. When anobject not influencing the movement of the carrier approaches, radarsstill drive the warning unit to submit warning messages, inducingadditional bother to drivers. Although the detection for surroundingobstacles of a carrier has been improved, in the moving process of thecarrier, there still exists danger caused by other carriers. Moreover,there are more objects that can influence driving safety. For example,pedestrians, animals, and moving objects can be regarded as theobstacles for moving carriers. They will cause emergency situations inthe moving process. The influence is worst in the crowded streets ofcities.

Dash cams are developed to record color images under emergencysituations for judgements afterwards. Unfortunately, they do not solvethe root problem. To solve the problem, drivers should be able toprevent emergency situations. Current dash cams are disposed on thefront and rear sides of a carrier. There still exist blind spots on bothsides. It is required to have image equipment integrated with detectiontechnologies for both sides for further assisting drivers to preventblind spots. In addition, according to the detection for the lateralblind spots, dangers can be predicted and drivers can be notified forprotecting them.

Dangerous situations of carriers will not occur only at crossroads. Theywill happen even in parking, especially when auto parking technologiesare widely applied. ADAS alone is not sufficient to protect drivers. Theprediction of dangers is also required.

Accordingly, the present application provides a system for sensing andresponding to a lateral blind spot of a mobile carrier and the methodthereof. By scanning the objects on one side of a mobile carrier, thecorresponding object image will be given. Then the images are filteredto give filtered images that indicate influence on the carrier.According to the corresponding objects in the filtered images, the pathsof the objects will be predicted. By modifying the moving route, thedangerous situations can be avoided.

SUMMARY

An objective of the present application is to provide a system forsensing and responding to a lateral blind spot of a mobile carrier andthe method thereof. By scanning the objects on one side of a carrier,the corresponding object image will be given. Then the images arefiltered to give filtered images corresponding to the lateral blind spotof the mobile carrier. According to the corresponding objects in thefiltered images, the paths of the objects will be predicted. Bymodifying the moving route, the dangerous situations can be avoided.

To achieve the above objective, the present application discloses amethod for sensing and responding to a lateral blind spot of a mobilecarrier. The mobile carrier includes a host connected to a lightscanning unit and an image extraction unit. The host executes the stepsof the method. First, the host executes a parking command correspondingto the mobile carrier for enabling the mobile carrier to park to thecorresponding parking space. The host generates a positioning messageaccording to the relative location or absolute location of the mobilecarrier with respect to the parking space. Next, the host generates afirst moving route according to the positioning message and the parkingspace. The light scanning unit scans one or more object at the parkingspace according to the first moving route. The image extraction unitextracts one or more object image correspondingly. Then, the host adoptsan image optical flow method to classify the one or more object imageand giving the corresponding one or more filtered image of the parkingspace. Afterwards, the host generates one or more predicted pathaccording to the corresponding object vector of the one or more filteredimage. Namely, the host predicts the path of the corresponding object ofthe one or more filtered image. Next, the host modifies the first movingroute according to the one or more predicted path and generates a secondmoving route correspondingly. In other words, the host performs dangerprediction on the blond spots of the mobile carrier and adjusts thecorresponding moving route of the mobile carrier. Accordingly, thepresent application can provide danger prediction on lateral blind spotsof a mobile carrier in the parking process and generates thecorresponding modified moving route. Then the driving assistance systemcan intervene driving control according to the notification message andnotifies the driver concurrently.

According to an embodiment of the present application, in the steps inwhich the light scanning unit scans one or more object at the parkingspace according to the first moving route and the image extraction unitextracts one or more object image correspondingly, the light scanningunit further scans the one or more object surrounding the parking spaceand the image extraction unit extracts the corresponding one or moreobject image surrounding the parking space.

According to an embodiment of the present application, in the steps inwhich the host adopts an image optical flow method to classify the oneor more object image, the host extracts a plurality of three-dimensionalimages according to the one or more filtered image and classifies theone or more object image using the image optical flow method accordingto the positioning message.

According to an embodiment of the present application, in the step inwhich the host modifies the first moving route according to the one ormore predicted path and generates a second moving route correspondingly,the host judges if a first effective area of the parking space is shrunkto a second effective area according to the one or more predicted path.The first effective area is greater than a carrier size of the mobilecarrier. The second effective area is smaller than the carrier size.When the first effective area is shrunk to the second effective area,the second moving route guides the mobile carrier to park to a portionof the parking space.

According to an embodiment of the present application, in the step inwhich the host modifies the first moving route according to the one ormore predicted path and generates a second moving route correspondingly,the host calculates according to a corresponding radius differencebetween inner wheels and a turning angle of the first moving route andthe one or more predicted path and then modifies the first moving routeand generates the second moving route correspondingly.

The present application further provides a system for sensing andresponding to a lateral blind spot of a mobile carrier and the mobilecarrier may set a host, a positioning unit, a light scanning unit, andan image extraction unit. The host is disposed in the mobile carrier.The light scanning unit and the image extraction unit are disposed onone side of the mobile carrier. The host executes a parking commandcorresponding to the mobile carrier for enabling the mobile carrier topark to the corresponding parking space. The host generates apositioning message according to the relative location or absolutelocation of the mobile carrier with respect to the parking space. Next,the host generates a first moving route according to the positioningmessage and the parking space. The light scanning unit scans one or moreobject at the parking space according to the first moving route. Theimage extraction unit extracts one or more object image correspondingly.Then, the host adopts an image optical flow method to classify the oneor more object image and giving the corresponding one or more filteredimage of the parking space. Afterwards, the host generates one or morepredicted path according to the corresponding object vector of the oneor more filtered image. Namely, the host predicts the path of thecorresponding object of the one or more filtered image. Next, the hostmodifies the first moving route according to the one or more predictedpath and generates a second moving route. In other words, the hostperforms danger prediction on the blond spots of the mobile carrier andadjusts the corresponding moving route of the mobile carrier.Accordingly, the present application can provide danger prediction onlateral blind spots of a mobile carrier in the parking process andgenerates the corresponding modified moving route. Then the drivingassistance system can intervene driving control according to thenotification message and notifies the driver concurrently.

According to an embodiment of the present application, the lightscanning unit is a lidar or a radar scanner.

According to an embodiment of the present application, the lightscanning unit further scans the one or more object surrounding theparking space and the image extraction unit extracts the one or moreobject image surrounding the parking space.

According to an embodiment of the present application, the host judgesif a first effective area of the parking space is shrunk to a secondeffective area according to the one or more predicted path. The firsteffective area is greater than a carrier size of the mobile carrier. Thesecond effective area is smaller than the carrier size. When the firsteffective area is shrunk to the second effective area, the second movingroute guides the mobile carrier to park to a portion of the parkingspace.

According to an embodiment of the present application, the hostcalculates according to a corresponding radius difference between innerwheels and a turning angle of the first moving route and the one or morepredicted path and then modifies the first moving route and generatesthe second moving route correspondingly.

According to an embodiment of the present application, the location ofthe lateral blind spot is a blind spot region corresponding to theparking space of the mobile carrier and defined by the intelligenttransport system ISO 17387.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flowchart according to an embodiment of the presentapplication;

FIG. 2A to FIG. 2F show schematic diagrams of partial steps according toan embodiment of the present application;

FIG. 3 shows a schematic diagram of perspective projection methodaccording to an embodiment of the present application;

FIG. 4 shows a schematic diagram of parking in a portion of the parkingspace according to an embodiment of the present application; and

FIG. 5 shows a schematic diagram of parking in a parking space accordingto an embodiment of the present application.

DETAILED DESCRIPTION

Since the radar system according to the prior art and dash cams cannotprovide prediction of lateral blind spots of a mobile carrier, thepresent application provides a system for sensing and responding to alateral blind spot of a mobile carrier and the method thereof foravoiding the dangerous situations caused by later blind spots of amobile carrier.

In the following, the properties and the accompanying system of themobile carrier warning sensor fusion system and the method thereofaccording to the present application will be further illustrated.

First, please refer to FIG. 1 , which shows a flowchart according to anembodiment of the present application. As shown in the figure, accordingto the method for sensing and responding to a lateral blind spot of amobile carrier of the present application, the host should execute thefollowing steps:

-   -   Step S10: Judging if the mobile carrier turns and moves to a        parking space;    -   Step S12: Generating a positioning message according to the        relative location or absolute location of the mobile carrier        with respective to the parking space;    -   Step S122: The host generating a first moving route according to        the positioning message and a location message of the parking        space;    -   Step S14: A light scanning unit scanning the corresponding        objects at or surrounding the parking space and an image        extraction unit extracting the corresponding object images;    -   Step S16: Classifying the object images using an image optical        flow method and giving the corresponding filtered images of the        first moving route;    -   Step S18: Generating a predicted route according to the        corresponding object vectors of the filtered images; and    -   Step S20: Adjusting the first moving route according to the        predicted route and generating a corresponding second moving        route.

Please refer to FIG. 2A to FIG. 2E, which illustrate the accompanyingidentification system 1 for the method for sensing and responding to alateral blind spot of a mobile carrier according to the presentapplication. The system 1 comprises a host 10, a light scanning unit 20,and an image extraction unit 30. The host 10 according to the presentembodiment is an automotive computer that includes a processing unit 12and a memory 14. Nonetheless, the present application is not limited tothe embodiment. The host 10 according to the present application can bea server, a notebook computer, a tablet computer, or any electronicdevice with image processing capability. The light scanning unit 20according to the present embodiment is a lidar or a laser scanner. Theimage extraction unit 30 according to the present embodiment is a colorimage extraction unit, for example, an automotive CMOS image sensor. Thehost 10 uses the processing unit 12 to execute an operational program Pfor receiving the image data IMG generated by the image extraction unit30 and performing image processing. The host 10 is disposed in a mobilecarrier V. The light scanning unit 20 and image extraction unit 30 aredisposed on one side of the mobile carrier V. The host 10 is connectedelectrically to the light scanning unit 20 and the image extraction unit30. An image extraction angle of the image extraction unit 30 accordingto the present embodiment is between 120 and 170 degrees and extractsobject images with 10 meters. Besides, the host 10 is further connectedelectrically to a positioning unit 40.

In the step S10, as shown in FIG. 2A, the host 10 judges if a parkingcommand CMD is executed. In other words, the host judges the mobilecarrier V turns and heads for a parking space 50. If not, the host 10continue to judge if there is a parking command by re-executing the stepS10. When there is a parking command CMD, the step S12 is executed.Please refer to FIG. 2A and FIG. 2B. According to the presentembodiment, a positioning message 42 generated by the positioning unit40 is transmitted to the processing unit 12 of the host 10. Thepositioning unit 40 generates the positioning message 42 to theprocessing unit 12 according to the absolute location of the mobilecarrier V and the parking space 50. Then the processing unit 12generates a corresponding first moving route L1 of the mobile carrier Vaccording to the positioning message 42 and the parking space 50 in thestep S122. For example, the first moving route L1 is the mobile carrierV turns and heads for the parking space 50. The first moving route L1 isa predetermined route for the mobile carrier V to move to the parkingspace 50. Thereby, according to the present embodiment, the step S14will be executed subsequently. In addition to using the positioning unit40 to provide the positioning message 42 of absolute location, the lightscanning unit 20 can perform optical scanning on one side of the mobilecarrier V or even 10 to 50 meters surrounding the mobile carrier V forproviding the positioning message 42 of relative location. In otherwords, the light scanning unit 20 acquires the positioning result forthe space surrounding the mobile carrier V and hence providing thepositioning message 42 corresponding to the parking space 50 withrespect to the mobile carrier V.

The host 10 executes the step S14. Please refer again to FIG. 2A andFIG. 2B. The host 10 uses the light scanning unit 20 to perform opticalscanning on one side of the mobile carrier V, especially on the parkingspace 50, according to the first moving route L1. It also scans thesurroundings of the parking space 50. Namely, the light scanning unit 20scans the objects corresponding to the parking space 50. The scanningmethod of the light scanning unit 20 is to generate one or more opticalgrating 22 to one or more object. According to the present embodiment,the objects includes a first object VO1 and a second object VO2, whichwill produce reflection light 32 according to the optical grating 22 tothe image extraction unit 30 and hence producing a plurality of objectimages OBJ correspondingly. According to the present embodiment, thelight scanning unit 20 is a lidar. A plurality of parallel stripes oflight, particularly, vertical laser light, form the optical grating 22.The image extraction unit 30 extracts the corresponding reflection light32 of the optical grating 22 and generates the corresponding objectimages OBJ of the reflection light 32. In addition, the light scanningunit 20 according to the present application can further be a laserscanner which achieves the effect of a lidar by a plurality of laserscans. The processing unit 12 executes the operational program P forprocessing the object images OBJ extracted by the image extraction unit30 and hence highlighting the object images OBJ corresponding to thefirst object VO1 and the second object VO2, as well as performing imagestitching or color and greyscale calibration on the object images OBJfor subsequent spatial identification.

The location of the lateral blind spot is a blind spot regioncorresponding to the parking space of the mobile carrier V and definedby the intelligent transport system ISO 17387. For the first object VO1or the second object VO2 in the blind spots, the light scanning unit 20and the image extraction unit 30 can assist to extract the unawareplaces. In addition, the ADAS also needs a more complete imageextraction for identifying lateral objects, such as pedestrians, cars,bus stops, traffic labels, or traffic lights, or even the A-pillars,which are the visual direction that always induces blind spots.

Next, in the step S16, as shown in FIG. 2C, the processing unit 12executes an image optical flow method L for filtering the object imagesOBJ and giving the filtered images IMG. In other words, the processingunit 12 filters the corresponding objects according the first movingroute L1 of the mobile carrier V and acquiring the correspondingfiltered images IMG. For example, if the object is an roadside object orcar, the processing unit 12 will not take its corresponding object imageOBJ into consideration and the corresponding object image OBJ will notbe labeled as one of the filtered images. As shown in FIG. 2B, theobject VO includes the first object VO1 and the second object VO2. Thesecond object VO2 is a roadside car and hence will not influence thefirst moving route L1 of the mobile carrier V. Thereby, the object imageOBJ of the second object VO2 will not be labeled as a filtered imageIMG. That is to say, the object image OBJ of the first object VO1 willbe filtered and become a filtered image IMG. The processing unit 12according to the present embodiment executes the operational program Pto extract a three-dimensional (3D) image V3D of the first object VO1and performs spatial identification according to the three-dimensionalimage V3D. Namely, the host 10 performs spatial identification accordingto the three-dimensional image V3D and uses the positioning message 42provided by the positioning unit 40 to confirm that the second objectVO2 is a parked car and not moved. In addition, the first object VO1according to the present embodiment is the person taking the mobilecarrier V. Nonetheless, the present application is not limited to theembodiment. Alternatively, the first object VO1 can be a moving car.

In the step S18, please refer to FIG. 2B and FIG. 2D, the host 10executes the operational program P and performs a prediction operationaccording to the filtered images IMG for predicting the predicted routeML corresponding to the first object VO1 of the filtered images IMG. Theprocessing unit 12 performs the prediction operation according to thepositioning message 42 and the corresponding object vectors of thefiltered images IMG to give the corresponding route data MLD of thefiltered images IMG. The route data MLD correspond to the predictedroute ML shown in FIG. 2B. The corresponding object vectors of thefiltered images IMG can be a zero vector, representing a stationaryobject influencing the first moving route L1.

In the step S20, please refer to FIG. 2B and FIG. 2E, the host 10executes the operational program P and refers to the first moving routeL1 of the mobile carrier V to give first moving data L1D, for example,the turning angle and the radius difference between inner wheels. Thefirst moving data L1D is calculated with the route data MLD given in thestep S18 to generate a second moving route L2. The host 10 will adjustthe first moving data L1D according to the route data MLD and henceadjusting the first moving route L1 of the mobile carrier V for furthergenerating second moving data L2D of the second moving route L2, forexample, delaying moving, changing the inserting angle of the mobilecarrier V into the parking space 50, or changing the parking space 50.In addition to displaying on a display unit (not shown in the figures)for notifying the driver of the mobile carrier V with the dangeroussituation at the blind spot on one side of the mobile carrier V, thesecond moving route L2 generated by the host 10 according to the presentapplication can be further applied to the ADAS for intervening drivers'the driving behaviors for avoiding danger.

The equations for calculating the radius difference between inner wheelsinclude:

$\begin{matrix}{\alpha = {\sqrt{R^{2} - L^{2}} - \frac{d_{2}}{2}}} & (1)\end{matrix}$ $\begin{matrix}{{\cos\alpha} = \frac{a + \frac{d_{2}}{2}}{R}} & (2)\end{matrix}$ $\begin{matrix}{b = \sqrt{R^{2} + \left( \frac{d_{2}}{2} \right) - {d_{1}R\cos\alpha}}} & (3)\end{matrix}$ $\begin{matrix}{m = {b - a}} & (4)\end{matrix}$

R is the turning radius of the mobile carrier V; L is the wheelbase; d₁is the distance between front wheels; d₂ is the distance between rearwheels; α is the angle between the midpoint of the front and rear axlesof the mobile carrier V and the center of the turning circle; a is themoving radius of the central line of the inner rear wheel; b is themoving radius of the central line of the inner front wheel; and m is theradius difference of inner wheel of a non-trailer carrier.

As shown in FIG. 3 , by using the perspective projection method, theimage point P₀ extracted by the image extraction unit 30 includes afirst image point P₁ and a second image point P₂. The coordinates (x, y)of the first image point P₁ are located in the first domain DM1; thecoordinates (x′, y′) of the second image point P₂ are located in thesecond domain DM2. Thereby, the relation between the first image pointP₁ and the second image point P₂ extracted by the image extraction unit30 can be expressed by the following equations:

$\begin{matrix}{x^{\prime} = \frac{{m_{0}x} + {m_{1}y} + m_{2}}{{m_{6}x} + {m_{7}y} + 1}} & (5)\end{matrix}$ $\begin{matrix}{y^{\prime} = \frac{{m_{3}x} + {m_{4}y} + m_{5}}{{m_{6}x} + {m_{7}y} + 1}} & (6)\end{matrix}$

(x,y) is the first image point P₁; (x′, y′) is the second image pointP₂; m₀, m₁, . . . m₇ are the parameters relevant to the image extractionunit 30, including the focal length, the turning angle, and sizingparameters. The image points can be expanded to a plurality of imagepoint pairs. Then the Levenberg-Marquardt algorithm can be used toperform nonlinear minimization and giving the optimum values of m₁ tom₇, which is used as the optimum focal length for the image extractionunit 30.

The above image optical flow method L adopts the Lucas-Kanade opticalflow algorithm for estimating obstacles. The image difference method isused first. Then the image constraint equation is expanded by the Taylorequation:

$\begin{matrix}{{I\left( {{x + {\delta x}},{y + {\delta y}},{z + {\delta z}},{t + {\delta t}}} \right)} = {{I\left( {x,y,z,t} \right)} + {\frac{\partial I}{\partial x}\delta x} + {\frac{\partial I}{\partial y}\delta y} + {\frac{\partial I}{\partial z}\delta z} + {\frac{\partial I}{\partial t}\delta t} + {H.O.T.}}} & (7)\end{matrix}$

where H.O.T. means higher order terms in the equation and can beneglected for infinitesimal displacement. According to the equation, wecan get:

$\begin{matrix}{{{\frac{\partial I}{\partial x}\delta x} + {\frac{\partial I}{\partial y}\delta y} + {\frac{\partial I}{\partial z}\delta z} + {\frac{\partial I}{\partial t}\delta t}} = {0{or}}} & (8)\end{matrix}$ $\begin{matrix}{{{\frac{\partial I}{\partial x}\frac{\delta x}{\delta t}} + {\frac{\partial I}{\partial y}\frac{\delta y}{\delta t}} + {\frac{\partial I}{\partial z}\frac{\delta z}{\delta t}} + {\frac{\partial I}{\partial t}\frac{\delta t}{\delta t}}} = 0} & (9)\end{matrix}$

and giving:

$\begin{matrix}{{{\frac{\partial I}{\partial x}V_{x}} + {\frac{\partial I}{\partial y}V_{y}} + {\frac{\partial I}{\partial z}V_{z}} + \frac{\partial I}{\partial t}} = 0} & (10)\end{matrix}$

V_(x), V_(y), V_(z) are formed by x, y, z in the optical flow vectorI(x,y,z,t).

$\frac{\partial I}{\partial x},\frac{\partial I}{\partial y},\frac{\partial I}{\partial z},{{and}\frac{\partial I}{\partial t}}$

are the partial derivatives of the image with respective to thecorresponding directions at the point (x,y,z,t). Thereby, equation (10)can be converted to the following equation:

I _(x) V _(x) +I _(y) V _(y) +I _(z) V _(z) =−I _(t)  (11)

Rewriting equation (11) as:

∇I ^(T) ·{right arrow over (V)}=−I _(t)  (12)

Since equation (10) contains three unknowns (Vx,Vy,Vz), the subsequentalgorithm can solve for the unknowns.

First, assume that the optical flow vector (V_(x), V_(y), V_(z)) isconstant in a small m*m*m (m>1) cube. Then, according to the voxel 1 . .. n, n=m³, the following equation set will be given:

$\begin{matrix}{{{I_{{x}_{1}}V_{x}} + {I_{y1}V_{y}} + {I_{z_{1}}V_{z}}} = {- I_{t_{1}}}} & (13)\end{matrix}$ I_(x₂)V_(x) + I_(y2)V_(y) + I_(z₂)V_(z) = −I_(t₂) ⋮I_(x_(n))V_(x) + I_(yn)V_(y) + I_(z_(n))V_(z) = −I_(t_(n))

The above equation contain three unknowns and form an overdeterminedequation set, meaning there is redundancy therein. The equation set canbe represented as:

$\begin{matrix}{{\begin{bmatrix}I_{x1} & I_{y1} & I_{z1} \\I_{x2} & I_{y2} & I_{z2} \\ \vdots & \vdots & \vdots \\I_{x_{n}} & I_{y_{n}} & I_{z_{n}}\end{bmatrix}\begin{bmatrix}V_{x} \\V_{y} \\V_{z}\end{bmatrix}} = \begin{bmatrix}{- I_{t_{1}}} \\{- I_{t_{2}}} \\ \vdots \\{- I_{t_{n}}}\end{bmatrix}} & (14)\end{matrix}$

Denote (14) as:

A{right arrow over (v)}=−b  (15)

To solve this overdetermined problem, equation (15) adopts the leastsquare method to give:

A ^(T) A{right arrow over (v)}=A ^(T)(−b)  (16)

{right arrow over (v)}=(A ^(T) A)⁻¹ A ^(T)(−b)  (17)

We can get:

$\begin{matrix}{\begin{bmatrix}V_{x} \\V_{y} \\V_{z}\end{bmatrix} = {\begin{bmatrix}{\sum I_{x_{i}}^{2}} & {\sum{I_{x_{i}}I_{y_{i}}}} & {\sum{I_{x_{i}}I_{z_{i}}}} \\{\sum{I_{x_{i}}I_{y_{i}}}} & {\sum I_{y_{i}}^{2}} & {\sum{I_{x_{i}}I_{z_{i}}}} \\{\sum{I_{x_{i}}I_{z_{i}}}} & {\sum{I_{y_{i}}I_{z_{i}}}} & {\sum I_{z_{i}}^{2}}\end{bmatrix}^{- 1}\begin{bmatrix}{- {\sum{I_{x_{i}}I_{t_{i}}}}} \\{- {\sum{I_{y_{i}}I_{t_{i}}}}} \\{- {\sum{I_{z_{i}}I_{t_{i}}}}}\end{bmatrix}}} & (18)\end{matrix}$

Substituting the result of equation (18) into equation (10) forestimating acceleration vector information and distance information ofone or more object. Thereby, the one or more objects can be classifiedand their route can be predicted. For example, the object image OBJ ofthe first object VO1 is classified as a filtered image IMG, and thepredicted route ML of the first object VO1 is predicted.

In addition, as shown in FIG. 4 , the host can further get a firsteffective area A1 of the parking space 50 and a carrier size S, namely,the visual length and width, of the mobile carrier V. In the step S20,the processing unit 12 of the host 10 can judge if the first effectivearea A1 is shrunk to a second effective area A2. The first effectivearea A1 is greater than the carrier size S; the second effective area A2is smaller than the carrier size S. When the processing unit 12 of thehost 10 judges that the first effective area A1 is shrunk to a secondeffective area A2, the processing unit 12 adjusts the second moving dataL2D so that the second moving route L2 guides the mobile carrier V topark to a portion of the parking space 50. For example, one of the firstobjects VO1 is located on a side edge of the parking space 50 andshrinking the effective area of the parking space 50 to 80% and smallerthan the carrier size S. A portion of the mobile carrier V is located onor even exceeding the edge of the parking space 50. As shown in FIG. 5 ,when the processing unit 12 judges that the effective area of theparking space 50 is not changed, the processing unit 12 maintains thesecond moving data L2D and the second moving route L2 guides the mobilecarrier V to parking into the parking space.

To sum up, the present application provides a system for sensing andresponding to a lateral blind spot of a mobile carrier and the methodthereof. The host acquires the object images of a plurality of objectson one side of a mobile carrier for classifying and giving filteredimages. Then prediction calculations are performed on the correspondingobjects of the filtered images to give predicted route. The predictedroute is calculated with the moving route of the mobile carrier to givea second moving route. Besides, the host can further adjust the movingdata according to the route data for avoiding dangerous situations.

1. A method for sensing and responding to a lateral blind spot of amobile carrier, the mobile carrier comprising a host, a light scanningunit, and an image extraction unit, said host connected electrically tosaid light scanning unit and said image extraction unit, and said hostexecuting the following steps of: said host generating a positioningmessage according to the relative location or absolute location of saidmobile carrier with respective to a parking space when said hostexecutes a parking command according to a corresponding parking space onone side of said mobile carrier; said host acquiring a first movingroute to said parking space according to said positioning message and acorresponding location message of said parking space; said lightscanning unit scanning the corresponding one or more object at theparking space, said image extraction unit extracting the correspondingone or more object image, and said one or more object corresponding to alateral blind spot of said mobile carrier; classifying said one or moreobject images using an image optical flow method and giving thecorresponding one or more filtered image of said first moving route;generating one or more predicted route according to the correspondingone or more object vector of said one or more filtered image; andmodifying said first moving route according to said one or morepredicted route and generating a second moving route correspondingly. 2.The method for sensing and responding to a lateral blind spot of amobile carrier of claim 1, where in said step in which said lightscanning unit scans one or more object at said parking space accordingto said first moving route and said image extraction unit extracts oneor more object image correspondingly, said light scanning unit furtherscans said one or more object surrounding said parking space and saidimage extraction unit extracts t said corresponding one or more objectimage surrounding said parking space.
 3. The method for sensing andresponding to a lateral blind spot of a mobile carrier of claim 1, wherein said step in which said host adopts an image optical flow method toclassify said one or more object image, said host extracts a pluralityof three-dimensional images according to said one or more filtered imageand classifies said one or more object image using said image opticalflow method according to said positioning message.
 4. The method forsensing and responding to a lateral blind spot of a mobile carrier ofclaim 1, where in said step in which said host modifies said firstmoving route according to said one or more predicted path and generatesa second moving route correspondingly, said host judges if a firsteffective area of said parking space is shrunk to a second effectivearea according to said one or more predicted path; said first effectivearea is greater than a carrier size of said mobile carrier; said secondeffective area is smaller than said carrier size; and when said firsteffective area is shrunk to said second effective area, said secondmoving route guides said mobile carrier to park to a portion of saidparking space.
 5. The method for sensing and responding to a lateralblind spot of a mobile carrier of claim 1, where in said step in whichsaid host modifies said first moving route according to said one or morepredicted path and generates a second moving route correspondingly, saidhost calculates according to a corresponding radius difference betweeninner wheels and a turning angle of said first moving route and said oneor more predicted path and then modifies said first moving route andgenerates said second moving route correspondingly.
 6. A system forsensing and responding to a lateral blind spot of a mobile carriercomprising: a host, disposed in said mobile carrier, executing a parkingcommand according to a corresponding parking space of one side of saidmobile carrier, and generating a positioning message according to therelative location or absolute location of said mobile carrier withrespective to a parking space; a light scanning unit, disposed on saidside of said mobile carrier, scanning the corresponding one or moreobject at said parking space according to a first moving route, and saidone or more object corresponding to a lateral blind spot of said mobilecarrier; and an image extraction unit, disposed on said side of saidmobile carrier and adjacent to said light scanning unit, connectedelectrically to said host, and extracting the corresponding one or moreobject image of said one or more object; where said host executes animage optical flow method according to said first moving route forfiltering said one or more object image and giving one or more filteredimage; said host generates one or more predicted route according to oneor more object vector of said one or more filtered image; and said hostmodifies said first moving route according to said one or more predictedroute and generates a second moving route correspondingly.
 7. A systemfor sensing and responding to a lateral blind spot of a mobile carriersystem of claim 6, wherein said light scanning unit is a lidar or aradar scanner.
 8. A system for sensing and responding to a lateral blindspot of a mobile carrier system of claim 6, wherein said host judges ifa first effective area of the parking space is shrunk to a secondeffective area according to said one or more predicted path; said firsteffective area is greater than a carrier size of said mobile carrier;said second effective area is smaller than said carrier size; and whensaid first effective area is shrunk to said second effective area, saidsecond moving route guides said mobile carrier to park to a portion ofsaid parking space.
 9. A s system for ensing and responding to a lateralblind spot of a mobile carrier of claim 6, wherein said host calculatesaccording to a corresponding radius difference between inner wheels anda turning angle of said first moving route and said one or morepredicted path and then modifies said first moving route and generatessaid second moving route correspondingly.
 10. A system for sensing andresponding to a lateral blind spot of a mobile carrier of claim 6,wherein the location of the lateral blind spot is a blind spot regioncorresponding to said parking space of said mobile carrier and definedby the intelligent transport system ISO 17387.